1. Field of the Invention
The invention is employed in data processing systems which have a hierarchy of levels. Programs executing at one level employ functions provided by the next level down, and can thus use the resources of the lower level without knowledge of the implementation details of that level.
2. Description of the Prior Art
Data processing systems are often organized as a hierarchy of levels. At the lowest level, there is the physical hardware which performs the actual operations of moving and manipulating data; at a higher level is an operating system. The operating system controls the actual operation of the physical hardware and makes the resources of the hardware available to users of the operating system. In so doing, the operating system effectively defines a set of logical operations for higher levels in the hierarchy. Programs using this set of logical operations can run on any hardware which the operating system will run on. The data processing system may additionally include levels above the operating system which define logical operations for still higher levels. Examples of such levels include data base systems, transaction processing systems, and so on. Seen from the point of view of a given level, lower levels are system levels, because they define the set of logical operations used by programs running at the given level. Similarly, the operations provided by lower levels are system operations. The reverse is also true: seen from the point of view of a given level, higher levels are application levels, because they use the system functions defined by the lower level, and programs running at the application levels are application programs.
A great advantage of such hierarchical systems is that a given level hides the details of the operations it performs from higher levels. For example, the operating system defines a set of logical file operations; higher levels can use these logical file operations with no knowledge whatever of the manner in which the operating system actually performs the file operations on physical devices such as disk drives or terminals. As data processing systems and programs grow in complexity, hiding becomes more and more important: not only does it make it possible for programmers working at each level to master the complexity of the system, it also makes it possible to modify a lower level of the system without changing the higher levels.
A difficulty with hiding is that it works both ways: it not only hides the details of lower-level operations from higher levels, but also hides the details of higher-level data structures from the lower-level operations. For example, an application program may wish to construct a tree data structure in memory; to do that, it uses a system allocate function, which typically takes two arguments: a pointer to the allocated memory and a specification of the size of the allocated memory. Since these are the only arguments, the allocate function necessarily has as little notion of the tree data structure which it is being used to construct as the application program has of the memory system in which the space is being allocated. In consequence, the application program cannot simply use the system allocate function to allocate the tree data structure; instead, it must use the system allocate function to allocate raw memory space and then use an initialization routine running at the application level to make the space provided by the allocate function into the tree.
An area in which the hiding of higher-level data structures from lower-level operations causes particular difficulty is in distributed systems. In a distributed system, the data is processed by a set of component systems connected by a network. The component systems may have different hardware and/or operating systems and may send and receive data using different transfer syntaxes. When data is sent from a source component system to a destination component system in such a distributed system, it is often necessary to put the data in a form which permits the data to be distributed across the network, convert the data into the format required for the destination component's hardware or operating system, and put the converted data into the transfer syntax required for the destination machine. From the point of view of the application program, the send and receive operations are clearly system operations; on the other hand, the operations of putting the data in a form which permits distribution across the network, converting the data into the formats required by the destination machine, and putting the converted data into the proper transfer syntax all require a knowledge of the data which is available only at the application program level.
One way in which the prior art has attempted to deal with the fact that the form of the data being sent or received is hidden from the system-level send and receive operations is shown in the Open Network Computing system developed by Sun Microsystems. As presented in John Corbin and Chris Silvieri, "Open Network Programming", Unix World, December 1989, pp. 115-128, the Open Network Computing system uses remote procedure calls to send data to and receive data from other components of a distributed system. Data being sent is converted into a transfer syntax called External Data Representation (XDR). The Open Network Computing System provides library routines for converting integers, Booleans, enumerations, floating point values, bytes, arrays, character strings, structures, and unions to and from XDR; the user can use these library routines to write conversion procedures. Addresses of these conversion procedures are then used as arguments in the remote procedure calls.
While the Open Network Computing system does overcome the consequences of hiding the form of the data being sent or received from the system send and receive operations, it does so at a cost: first, the conversion to and from XDR is no longer hidden from the application program, but must be specified in every remote procedure call; second, only conversion to and from XDR are possible in the system; third, the conversion must be described within the framework of primitive and complex data types provided by XDR. What is lacking is apparatus and methods which permit both hiding of the details of the system operations and definition by the application program of portions of system operations which are affected by the application program's data, which do not restrict the application programmer to a single transfer syntax, and which do not require the application programmer to describe conversions using a built-in type framework. It is an object of the present invention to provide such apparatus and methods.